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ATTENTION DETECTION 

Field of the Invention 

[0001] The present invention relates to detection of an attention state 
of at least one animate object and particularly, although not exclusively, to a 
method and apparatus for autonomously collecting images. 

Background to the Invention 

[0002] Known camera technology, including cameras of the 
conventional film type, and digital cameras, consumer type cameras and 
professional type cameras, are based upon a usage model which requires a 
human photographer to point the camera at a field of view containing a subject of 
which a picture, image or video is to be captured, and activating a trigger button 
causing the camera to capture a still image, or in the case of a video camera, a 
moving picture scene. 

[0003] Known still image cameras and video/film cameras for 
consumer and professional use are effectively products which are used by a 
photographer to capture an image or moving sequence of a scene, where the 
photographer consciously selects the scene, sets up the camera positioning for a 
particular view, and activates capture of an image or sequence of images as a 
photographer controlled activity. 

[0004] In some cases, automatic triggers are provided on a timer 
device, whereby the photographer can set a camera up pointing at a scene, 
activate a time delay trigger, giving the photographer time to enter the scene, and 
then the camera automatically takes a picture or sequence once the pre- 
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determined time limit has elapsed, allowing the photographer to appear in her 
own images. 

[0005] There are also known prior art surveillance camera systems, 
which automatically capture still images or video sequences. Such cameras are 
used for security purposes in commercial premises such as factories and offices, 
as well as in domestic environments for security. These known cameras capture 
images or sequences of images in response to sensor activated events, such as 
an infra-red sensor being triggered, a movement sensor, or a circuit breaker 
sensor, for example a sensor which is activated when a door or window is 
opened. In general, this type of camera is wall mounted or mounted on posts, 
and pre-arranged to take a picture of a pre-determlned scene. The cameras are 
not mobile and provide a permanent security monitoring installation. 

[0006] In some cases, sun/eillance cameras are directable remotely 
from a cental console unit, so that they can pan across a field of view and focus 
in and out of that field of view by remote control, the cameras being moved and 
controlled by servomotors. 

[0007] There are also known surveillance cameras for other uses, for 
example traffic monitoring, and speed detection, which are triggered by motion 
sensors whidi detect vehicles moving within pre-set speed limits. This type of 
camera is conventionally installed in a permanent installation, pointing at a pre- 
determined scene. 

[0008] The known sun/eillance cameras and traffic monitoring 
cameras are at fixed installations which operate without a human photographer, 
but are limited in their fields of view. 

[0009] More recentiy, compact cameras have been installed in 
personal communication devices, such as mobile phones. Pictures of individuals 
can be taken using hand held devices, and sent as messages over a wireless 
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communication network to personal handsets of other users. With these hand 
held devices, the basic mode of usage requires a person to act as a 
photographer, pointing the device at a field of view, for example directing the 
device at their own face to take a picture of themselves, and activating capture of 
an image by triggering a switch. 

[OOtO] The concept of a user wearable camera device receiving 
attention clues from a host wearer and capturing images is known. For example 
in Summarizing Wearable Video, K. Aizawa. K. I. Ishijima, M. Stiiina. IEEE, 
International conference on image processing. 111:398-401. Thessaloniki. Greece, 
2001, there is disclosed a system which receives sensor inputs from a host 
wearer, and which attempts to understand those sensor inputs in order to 
determine when to capture an image. 

[0011] Other known worics which analyze a host wearer's attention 
from a self perspective point of view include the following: 

J. Healey and R.W. Picard, "StartleCam: A Cybernetic Wearable Camera", 
In Pmceedings of ttte International Symposium on Wearable Computers, pages 
42-49, 1998; 

A. Lockerd, F. Mueller, "LAFcam - Leveraging Affective Feedback 
Camcorder", ACM CHI. 2002; and 

Y. Nakamura, J.Ohde, Y. Otha. "Structuring personal activity records 
based on attention: Analysing vidoes from a head-mounted camera", in 
International Conference Pattern Recognition. Barcelona, September 2000. 

[0012] Known usage models and systems for detecting a person's 
attention focus on analyzing the actions and behavior of the person from the 
perspective of that person, that is, analyzing parameters which are local to the 
person and in the immediate vicinity of the person. 
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[0013] A further body of work concerned with observing a person from 
a position external of that person, that is from an extemal perspective includes 
the following items: 

A. Pentland, "Looking at People: Sensing for Ubiquitous and Wearable 
Computing. IEEE Trans. Pattem Analysis and Machine Intelligence, IEEE CS 
Press. Los Alamitos. Calif.. Jan. 2000, pp. 107-118; 

M. Trivedi, I. MIkic, S. Bhonsle, "Active Camera Networks and Sematic 
Event Databases for Intelligent Environments", IEEE Workshop on Human 
Modeling. Analysis and Synthesis. June 2000; 

A. Pentland, T. Choudhury. "Face Recognition for Smart Environments". 
IEEE Computer, vol. 33, no. 2, February 2000, pp. 50-55; 

D. Gavrila. 'The Visual Analysis of Human Movement: A Survey", 
Computer Vision and Image Understanding, vol. 73. no.1, January 1999. pp. 82- 
98: 

V. Pavlovic. R. Sharnia. T. Huang. "Visual Interpretation of Hand Gestures 
for Human-Computer Interaction: A Review". IEEE Trans. PAMI, vol. 19. no. 7. 
July 1997. pp. 677-695; 

R. Stiefelhagen, J. Yang, A. Waibel, "Estimating focus of attention based 
on gaze and sound". Proceedings of the Workshop on Perceptive User 
Interfaces, 2001; and 

R. Vertegaal. R. Slagter, G. van der Veer, A. Nijholt, "Why conversational 
agents should catch the eye". In Summary of ACM CHI Conference on Human 
Factors in Computing, The Hague, 2000. 

[0014] Some known devices such as disclosed in the above bodies of 
work could be refen-ed to as 's', which observe a person firom a position extemal 
of that person. 

Summary of the Invention 

[0015] A first aspect of the present invention relates to autonomous 
image capture, and the detection of situations of interest to users of cameras, in 
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order to trigger automated image capture without conscious human user 
activation of a camera device. Embodiment of the invention provide an 
autonomous wearable camera system which is able to capture Images of 
moments by infening situations of Interest to persons. 

[D016J Further embodiments may provide for the use of a detertor 
device which is able to Interpret 'attention dues' given out by human persons, the 
attention dues being used to activate one or more Image capture devices for 
capture of Images when the attention dues are detected. The device may be 
provided as one or more flmiware devices, or as a software agent. Specific 
embodiments herein aim to provide a system for autonomously detemilning a 
situational saliency based upon the attention dues. In order to trigger capture of 
one or a set of images of that situation. 

[0017] In one embodiment, a user carries a wearable camera. The 
detector device may be situated on the wearer's body. In other embodiments, the 
detector device may be situated elsewhere, or the detector device may be 
distributed. 



[0018] If the detector device is worn by a person canying a camera, 
the detector device may be capable of adopting a 'self-perspective' mode, in 
which the detector device observes situations from the perspective of a host 
person who is wearing an image capture device. In this mode, the detector device 
may receive attention clues in the form of sensor signals of parameters of the 
wearer, for example skin conductivity, body movement, or voice utterances made 
by the host wearer. In tiie self-perspective mode, since the detector device Is in 
dose proximity to a person. In order to capture Images in the self perspective 
mode, the detector device requires Inputs from sensors strategically placed on 
the host person, induding limb positioning sensors, eye trackers, and the like. 

[0019] The detector device combines information collected about the 
host wearer in ttie self perspective mode with externally received infomiation 
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collected about the same host wearer or similar or complimentary device which 
has collected information in an 'external' perspective mode. The detector device 
determines from the combination of self perspective attention clues and external 
perspective attention clues whether the host wearer is experiencing a situation in 
which his attention is raised to a level where recording of an environmental 
situation in which the user finds themselves would be of interest to that user. 

[0020] In an 'extemal* perspective mode, a detector device may 
analyze another person's body language and attention dues from a perepectlve 
external to the person, that is, as an entity remote from the person and obsen^ing 
the person. The detector device may be physically positioned away from that 
other person, and may observe that other person. In this mode, the detector 
device may detect externally detectable attention clues given out by the other 
person, for example JImb movements, pointing of a finger, movement of eye look 
direction, rapid re-orientation of a head of the person, aspects of the other 
persons body posture, and like attentfon indicating body language signals. 

[0021] According to one aspect of the present invention, there is 
provided an attention detection system comprising: 

a portable attention detection module capable of receiving attention clues 
of a host person, said attention dues generated from a self perspective of said 
host person, and receiving attention dues of said host person generated from an 
observer perspedive; 

at least one sensor device capable of generating signals relating to said 
attention clue signals of said self perspedive; and 

at least one sensor device capable of generating signals relating to said 
attention due signals of said observer perspective; 

wherein said attention detection module Is capable of detemiining a 
situation of raised attention of said host wearer from said self perspective 
attention dues, and said received obsen/er perspective attention dues. 



Docket No. 150&487 

-7- 

[0022] Other aspects of the invention are as recited in the claims 

herein. 



Brief Description of tlie Drawings 

[0023] For a better understanding of the invention and to show how the 
same may be carried into effect, there will now be described by way of example 
only, specific embodiments, methods and processes according to the present 
invention with reference to the accompanying drawings in which: 

[0024] Fig. 1 is a schematic diagram of one embodiment of a pereonal 
image capture device worn by a host person ; 

[0025] Fig. 2 is a schematic diagram of one embodiment of an image 
capture system comprising a plurality of Image capture devices, each worn by a 
corresponding host person; 

[0026] Fig. 3 Is a flow diagram of logical entitles of a specific image 
capture sy^em; 

[0027] Fig. 4 is a flow diagram of how infonnation Is processed in the 
image capture system of Fig. 3; 

[0028] Fig. 5 Is a block diagram of one embodiment of a people 
observer device; 

[0029] Fig. 6 is a block diagram of one embodiment of an attention 
detection module; 



[0030] 

capture device; 



Fig. 7 is a block diagram of one embodiment of an image 
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[0031] Fig. 8 is a schematic illustration of an exemplary subject with 
various sensor types according to the specific embodiment of the image capture 
system; 

[0032] Fig. 9 is a schematic illustration of a specific embodiment of an 
image capture system comprising two portable devices, each comprising an 
image capture device, a people observer device and an attention detection 
module; 



[0033] Fig. 1 0 is a schematic illustration of a two dimensional analyzed 
captured digital image of a person according to a specific method; 

[0034] Fig. 11 is a block diagram of a specific embodiment of an 
analyzer component of an attention detection module; 

[0035] Fig. 12 is a block diagram of sensor Inputs to a person 
observation device In a self perspective mode of operation of an Image capture 
system; and 



[0036] Fig. 13 is a flow diagram of the Interaction between first and 
second attention detection modules each operating in a self-perspective mode 
and in an observer mode. 



Detailed Description of the Drawing 



[0037] There will now be described by way of example a specific mode 
contemplated by the inventors for carrying out the invention. In the following 
description numerous specific details are set forth in order to provide a thorough 
understanding. It will be apparent however, to one skilled in the art. that the 
present Invention can be practised without limitation to these specific details. In 
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other instances, well known methods and structures have not been described in 
detail so as not to unnecessarily obscure the description. 

[0038] Fig. 1 is a schematic illustration of a person carrying a portable 
image capture device 100 capable of capturing images of a scene in a field of 
view of the image capture device. Typically, the image capture device 100 
comprises a digital camera having a charge coupled device (CCD) array. 

[0039] The image capture device 100 can be activated remotely by a 
wireless signal from a remote transmitter, in order to capture an image. The 
image capture device 100 can also be activated by signals generated by local 
sensors wom on the person's body. 

[0040] The portable image capture device 100 includes an attention 
detection module (ADM) which receives attention information indicating an 
attention level of the host wearer from a device extemal of the host wearer. 

[0041] The attention detection module communicates with at least one 
extemal people-observing device, such that the attention detection module 
combines self perspective information received directly from the host wearers 
body, with observer perspective infonnatlon conceming the host wearer, which is 
received from the extemal people-observing device, in order to detem^lne a 
condition under which the host wearer is experiencing a situation in which the 
wearer's attention is raised. In response to such a condition, the attention 
detection module activates capture of a still image, image sequence, or capture 
of another like media, from a host perspective. 

[0042] Whilst an image capture device 100 is described herein, with 
reference to the first embodiment of Fig.1 . attention clues of ttie host wearer can 
be used for capturing any ottier media type, for example including but not limited 
to audio data, still images, video sequences, pressure or temperature changes, or 
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other changes in environmental parameters which are experienced from the 
perspective of the host person. 

[00433 Fig. 2 Is a schematic illustration of how to use an image capture 
system in which two individual users 201 and 202 are in proximity to each other. 
Users 201 and 202 respectively carry portable image capture devices 203 and 
204. Each of devices 203 and 204 includes a detector device capable of 
observing a host person wearing the Image capture device, as well as observing 
other persons. The users 201 and 202 are close enough to each other such that 
(1) image capture device 203 of user 201 can capture a still or moving image of 
user 202. and (2) capture devices 203 and 204 can communicate with each other 
over a wireless link, for sending and receiving activation signals for activating 
capture of an image. 

[0044] In this situation, users 201 and 202 interact with each other 
using body language, such as waving, smiling, laughing, gesticulating, and/or 
adopting postures. Body language signals are collected through a system of 
sensors on devices 203 and 204. The sensors supply input attention signals into 
a people observing device (POD) comprising the detector device. The detector 
device converts the attention signals Into attention clue data. Devices 203 and 
204 process the attention due data obtained from the other person to detennine 
whether the body language of the other person indicates there is enough Interest 
in the first person, that capture of an image of a second person may be 
appropriate. If the saliency of the situation is high enough that capture of an 
image is appropriate, then one or more image capture signals Is generated for 
activating one or more image capture devices to capture one or more images. 

C0045] In the situation Illustrated in Fig. 2. a range of attention clues is 
collected, both from a self perspective, and from an external observer 
perspective, concerning the body language and behavior of the first host user 
201. Self perspective attention clues, for example body movement, breathing 
rate, heart rate, blood pressure, skin resitlvity. direction of motion and the like are 
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measured by sensors worn by the second host user 202. However other 
attention clues which are available from an observer perspective, but which are 
difficult to assess from a self perspective are provided by the attention detection 
module which is external of the host wearer of the image capture device. Such 
attention clue information include parameters such as overall body posture, eye 
gaze direction, facial expression, overall body motion, or limb motion such as 
waving, pointing location of the host wearer with respect to other objects in an 
environment or the like which are best observed from a position external of a 
person. 



[0046] The attention detection modules 203 and 204 earned or worn 
by the host users 201-202 assimilate both sets of attention information, that is. 
the self perspective and observer perspective attention dues, to determine a 
state of attention of ttie host user 201 or 202 of the attention detection module. 

[0047] Refemng to Fig. 3. there is shown schematically in a general 
case logical entities present in an image capture system accoitling to a specific 
embodiment of the present invention. The logical entities comprise a plurality of 
PODs 300. 301; a plurality of attention detection modules 302. 303; a plurality of 
image capture devices 304, 305; and a plurality of sensors 306, 307. 

[0048] In this example users 201 and 202 respectively carry PODs 
300. 301; an attention detection module (ADM) 302, 303; and an image captijre 
device 304. 305. However, in the general case, the location of ttie PODs. and 
attention detection modules need not be restiicted to being worn by human 
persons. Similarly, although in many embodiments sensors are worn by persons, 
in the general case, sensors can be located remotely from people, for example 
mounted on walls or posts, for observing persons remotely. 

[0049] Each modute 302. 303 comprises a set of algorithms for 
detecting attention dues exhibited by a host wearer of a particular image captore 
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system. Each peopfe observing device 300. 301 receives signals from one or a 
plurality of sensors 306, 307 respectively. 

[0050J Each attention detection module 302. 303 comprises one or 
more algorithms for analyzing attention clues relating to the host wearer of a 
particular Image capture device, and for analyzing attention dues received 
externally from ottier attention detection modules relating to attention clues 
exhibited by one or more other attention detection modules. 

[0051] Refening to Fig. 4. there is illustrated schematically infomiation 
processing stages shown in the image capture system of Fig. 3 herein. 

[0052] Body language Information 400 exhibited by the persons 
carrying or in proximity to an attention detection module is detected by one or 
more sensors of the module. The sensors generate sensor signals 401. which 
comprise signals, sensing various parameters, for example movement, sound. 2- 
dimensional images and the like. The sensor signals are supplied to an attention 
detection module (ADM), which analyses the sensor signals to detemiine a set of 
self attention clues 402 collected from a self perspective. Attention clues 
comprise data which indicate that the attention of the host wearer person has 
been raised. The attention clues are input into the attention detection module. 
The attention detection module also receives external attention clues 403 relating 
to the host wearer person as generated by an external people-observing device. 
The attention detection module processes the attention clue data to detemiine 
whether the attention level of the host wearer person is at a level where an image 
of a situation is likely to be of interest, and worth capturing. If so. the attention 
detection module generates a set of camera tiigger signals 404 for activating 
capture of images by one or more image capture devices. This results in 
capturing with a camera a set of images 405 of a situation that has occurred and 
has raised the attention level of one or more persons wearing attention detection 
modules to a sufficient level such that the situation triggers capture of the image. 
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[G053] Fig. 5 is a block diagram of components of a people-observing 
device 500. The people-observing device 500 comprises a transponder 501 and 
an antenna 502; a controller 503 for controlling the device 500; optionally, an 
image capture device 504 (e.g. a still or movie camera, preferably of the digital 
type) for capturing images of scenes; an image memory 505 for storing image 
data of images captured by the image capture device 504; a proximity sensor 506 
for sensing when the people-observing device 500 is within range of one or more 
other people-observing devices and/or one or more attention detection modules; 
a host sensor interface 507 for receiving signals from a plurality of host sensors 
which may be worn by a host person wearing the people-observing device; a set 
of connectors 508 which can be used optionally for directly connecting one or 
more sensors to the host sensor interface; and an attention detector 509 for 
detecting attention clues of a host wearer, and/or one or more other persons. 

[0054] First and second people-observing devices 500 can be 
provided with beacons to locate and detect each other, for enabling an exchange 
of information between the first and second people-observing devices. People- 
observing device 500 can be configured for recognizing at least one other people- 
observing device to form a group of peopleK)bserving devices which recognize 
each other. The people-observing device 500 is configured to communicate or 
not communicate with at least one other people-observing device, based on an 
analysis of contextual information relating to a host person, or an observed 
person. Such contextual infomiation can comprise the location of a person, time 
of day, a local environment where the person finds himself, or like contextual 
information. 



C0055] The people-observing device 500 can be configured to 
communicate or not communicate with at least one other people-observing 
device based on an analysis of a type of attention clue detected; for example, 
whether a short casual glance is exchanged with a person, or whether a longer 
glance is exchanged with a person. 
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[0056] The people-observing device 500 can also be set to 
communicate or not communicate with at least one of the attention detection 
modules depending upon the type of attention clue detected; for example a 
particular facial expression, a body posture type, a head direction, an eye gaze 
direction or an ami gesture. 

JOOST] Fig. 6 is a block diagram of components of attention detection 
module (ADM) 600. functionally illustrated in Fig. 4. Module 600 comprises a 
transponder 601 and an antenna 602 for communicating with (1) one or more 
other attention detection modules, and (2) one or more people-observing devices; 
a controller 603; a host peopleK)bserving device interface 604 for interfacing with 
the people-observing device worn by the same host person who is wearing the 
attention detection module; a self perspective attention analyzer 605 for analyzing 
attention dues of the host wearen an observer perspective analyzer 606 for 
analyzing attention clues observed of persons other than the host wearer person; 
and an internal data bus 607 connecting the above elements. It will be 
appreciated by persons skilled in the art that power supplies, casings, and like 
known components are also provWed to create a practical manufactured unit. 



The attention analyzers 606 and 606 comprise various 
components for detecting when the attention of a person or persons is raised. 
These components in analyzers 605 and 606 include components for detecting 
(1) facial expression of one or more persons. (2) eye direction of a person. (3) 
movement attitude of one or more persons. (4) body posture of one or more 
persons, and (5) context of an environment containing one or more persons. 

[0059] In the foregoing embodiments, an attention detection module 
has been shown as being worn personally by a host wearer. The attention 
detection module is personal to a host wearer, in the sense tiiat an attention 
detection module monitors (1) a host wearer and receives attention clues which 
have been collected fiiom the perspective of the host wearer, and (2) attention 
clues conceming that host wearer which have been collected from the 
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perspective of an observer who is not the host wearer. The physical location of 
the attention detection module is not restricted to being worn or personally carried 
by a host wearer; the module can be remote from the host wearer. However, an 
attention detection module is assigned to a host wearer for detecting an attention 
status of that host wearer, and therefore in that sense is personal to that host 
wearer. However in a prefenned embodiment, an attention detection module is 
canried or worn by a host wearer. 

[0060] Fig. 7 is a block diagram of components of image capture 
device 700. Image capture device 700 comprises a transponder 701 and an 
associated antenna 702; a set of image capture optics and sensors 703. for 
example a charge coupled device (CCD) array; a controller 704; an image 
memory 705 for storing image data; an intemal bus 706; and a people-observing 
device (POD) interface 707. for connecting to a local people^bserving device. 
The local POD and the image capture device are wom by the same host user. 

[0061] The image capture device 700 is capable of receiving trigger 
signals from one or more attention detection modules, over a wireless link using 
tfie antenna 702 and transponder 701. Activation of the Image capture 700 
device causes ttie device to capture an image or image sequence. Device 700 
stores the captured image or image sequence in the image memory 705 under 
control of the controller 704. 



[0062] The image capture device 700 can also be a monitoring sensor 
for collecting 2-dimensional images of one or more persons, for input into a 
people-observing device. In the embodiments shown, the capture image data 
can be transmitted over ttie wireless link via tiie antenna 702 and transponder 
701 to a remote people-observing device. 

[0063] It will be appreciated by ttie person skilled in the art that 
alttiough a peopje-observlng device 500. attention detection module 600. and an 
image capture device 700 are illustrated in Figs. 5 to 7 herein as stand alone 



Docket No. 1509-487 



-16- 

discrete entities, each capable of communicating with each other via a wireless 
linl<. these three components can be Integrated into a single physical device, or 
combinations of the components can be Integrated Into one or more physical 
devices. If the components are Integrated Into a single physical device, such 
single physical device can be worn by a single host person. 

C0064] In an embodiment where the people-observing device and the 
image capture device are both worn by a wearer, the image data can be made 
available directly to the people-observing device via an internal bus or an 
Interface. 



[0065] Fig. 8 is a schematic illustration of a host person wearing a set 
of personal sensors for detecting body language exhibited by that host person. 
Sensor types include the following: 

[00661 Motion sensors for detecting the motion of a limb of the host 
person, for example raising of an ami. rotation of the head or the like. The 
motfon sensor can take the fonn of a wrist strap 800 containing a small 
gyroscope, for example a laser gyroscope, or a fibre optic gyroscope. 
Altematlveiy. the motion sensor can be in the fonm of a jewelry item, for example 
an eamng or ear stud 801 for detecting rapid head rotation. The motion sensor 
can also include an accelerometer. Motion sensors can also be wom or attached 
to other clothing items, for example included in the sole of a shoe or other 
footwear. 



[0067J Temperature sensors: Temperature sensors for measuring skin 
temperature or other body temperature can be jewelry Items wom In contact with 
the skin, for example as a wrist wateh or bracelet. Temperature sensors can be 
included within clothing wom next to a person's skin, for example as thin 
conductive strips woven into material which a person wears. 
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[0068] Voice detectors and sound detectors. A sound detector can be 
provided in the people-observing device itself, or worn externally as a headset 
802 having a microphone positioned near the mouth of the host person. 

[0069] Two-dimensional image sensors: An image capture device 
itself, I.e.. camera 803 can be a sensor. For example the image capture device 
can capture an image of a person other than the host person wearing the image 
capture device, or a general scene external to a host person wearing the image 
capture device. 



[0070] There is now described in further detail various elements of the 
image capture system. In some cases, the elements described are specific to a 
particular embodiment of the image capture system, and it will be appreciated by 
persons skilled in the art that variations in the precise arrangement of the various 
elements described can be made. 



[0071] Refemng to Fig. 9 herein, there is shown schematically an 
image capture system comprising two portable camera units 900 and 902. Each 
of camera units 900 and 902 includes an image capture device; a people- 
observing device: and an attention detection module, included in a single portable 
and battery powered casing that can be worn, earned or held by a person. 

[0072] There are now described various aspects of operation of the 
image capture system described vwth reference to Figs. 1 - 9. 

[0073] Fig. 10 is an illustration of a still image captured by an Image 
capture device, from which an attention clue can be abstracted. The image of 
Fig. 10 includes a person having a facial expression, i.e.. a smile or laugh. The 
image capture device captures a 2-dlmensional image, which is processed by 
attention detection module 600, 
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[0074] The two dimensional Image is typically converted into a bitmap. 
Algorithms are applied to the bitmap for (1) edge detection to detect edges of 
features, and (2) detection of human faces or human outlines. These algorithms 
can be resident in either a peopIeK)bservlng device or an attention detection 
module. Facial expressions can be detected, by (1 ) detecting a set of edges, and 
then (2) comparing the set of detected edges, or points on those edges, with a 
predetermined library of edges or points corresponding to a person's smiling or 
laughing facial expression. 

[0075] The attention detection module comprises a self perspective 
attention analyzer and an observer perspective attention analyzer. Refemng to 
Fig. 11 herein, the self perspective and observer perspective attention analyzers 
can share attention detection components, such as common image processing 
components for processing two dimensional Images. The image processing 
components include edge detection algorithm 1101 for detecting edges within a 
two dimensional image; a human outline detector component 1102 for detecting 
a human outline in a two dimensional image frame, or a sequence of two 
dimensional image frames; a facial expression analysis component 1103 for 
detecting various facial expressions, for example dilated pupils, a smile, a 
surprise expression and like expressions which indicate that the attention of a 
human has been noticed by a subject; a motion analyzer component 1104 for 
analyzing motion from a sequence of two dimensional images; and a posture 
analyzer component 1 105 for analyzing the posture of a human being from a two 
dimensional image or a sequence of two dimensional images. 

10076] Each of the components of Fig. 11 can be provided by (1) a 
general purpose computer, under control of an application, or (2) finnware. for 
example in an application specific integrated circuit (ASIC). In which case the 
analyzer component can be made small enough to be portable and worn by a 
host human. 
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[0077] The attention detection module, in detemiining whether a 
person's state of attention has been raised sufficiently to activate capture of data 
on behalf of that person, takes into account attention dues collected from a self 
perspective, and attention clues detected from an observer perspective. 
However, the attention detection module does not necessarily need to give each 
type of attention clue an equal weighting, in determining whether a host person's 
attention has been raised. Depending upon the type of attention clue, and ottier 
factors, including for example an environment in which the host person finds 
himself, a disproportional higher amount of weighting can be given either to self 
perspective attention dues, or obsen/er perspective attention dues. 

[0078] Fig. 12 is a schematic diagram of a set of sensors feeding 
instmctions to a people observation device. The sensors can be incorporated 
integrally with the people observation device, or be provided externally to the 
people observation device, and send signals to tiie people observation device. 
Sensor t^>es include tiie following: 

temperature sensors for sensing skin temperature of a host person 
earring the sensor; 

an Image capture device, for example a charge couple delay device for 
caphJring Images in a self perspective mode of a scene, which can typically 
indude other people; 

a set of biosensors. Induding a heart rate monitor, a perspiration 
monitoring sensor, a skin conductivity monitoring senson 

one or more motion sensors for sensing parameters such as head 
rotation, ami movement and leg movement; and 

one or more audio sensors, for example a microphone. 

[0079] Each of tfie sensors feeds sensor signals to the people 
observation device. The people observation device processes tiie sensor 
signals, into a form to enable Vne signal to be analyzed by ttie attention detection 
module. 
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[0080] The detector device of the image capture system can be 
operated in various modes of operation including a self perspective mode, or an 
observer perspective mode. Each of these modes of operation requires a 
different configuration of the components of ttie system. 



[0081] The modes function as follows: 



[0082] In the self perspective mode of operation, the attention of a 
person Is activated from the perspective of a host person carrying an image 
capture device. The host person may become interested in various aspects of 
his environment. For example, the person may look at an airplane in the sky. or 
may turn around to look at a passing person, or may have his attention attracted 
by an event, for example a load bang, a car crash or some other distracting event 
in the person's immediate sunoundlngs. Sensors camed by ttie person monitor 
the host person's body language. A people-obsen^lng device carried by the host 
person receives signals from the sensors, and generates attention clue signals. 
The attention due signals comprise signals from tfie sensors at various levels of 
data processing. For example a two dimensional image data might have very 
little processing, and may undergo almost no processing of the infomiation 
content of the signal prior to being forwarded to the attention detection module. 
Infomriatlon In an audio signal may be processed before being presented to the 
attention detection module. For example ttie audio signal may be filtered such 
that only audio signals above a predetemiined amplitude are passed to the 
attention detection module, witti nomial background environment signals at a 
relatively low level of loudness being filtered out by ttie people-obsen^ng device. 
The level of infonnation processing to be applied to ttie sensor signals before 
ttiose signals are passed to ttie attention detection module is a function of a 
specific design parameter which can be varied from system to system. 

[0083] In ttie self perspective mode, the attention detection module 
receives signals firom ttie host person. The signals from the host person 
represent ttie body language of ttie host person, to which the attention detection 
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module is assigned to monitor, that is. represent information describing an 
attention level from the perspective of the host person. The ADM analyzes these 
self perspective attention clues. As a result of the analysis by the ADM. the ADM 
generates an image capture signal for activating an image capture device worn 
by the host person when the ADM determines that the attention of the host 
person Is raised to such a level that there Is a significant or Interesting event In 
the host person's Immediate environment to capture an image in the 
environment. 

[0084] In an observer perspective mode, one or more people- 
observing devices observe persons other than the host person wearing the 
people-observing device. For example where first and second host persons each 
cany first and second conresponding respective people-observing devices, and 
each canies a corresponding attention detec^on module, a first peopleobserving 
device of a first host person might observe the second person. Observation can 
take one of the fomns mentioned above, for example continuous monitoring of a 
moving captured image of the second person. Upon detecting a significant 
sensed parameter, the first people^bsenrfng device generates an attention clue 
signal which is processed by the first attention detection module carried by the 
host person. Attention dues relate to the second person, rather than the first 
host person. 

[0085] The first attention detection module, having determined that the 
second person's level of attention is raised to a high level, sends signals to the 
first image capture device wom by the first person, and/or an activation signal to 
the second Image capture device wom by the second person. The signals cause 
either one or both of those devices to capture a conesponding respective image. 

[0086] Conversely, the second people-observing device wom by the 
second person monitors ttie attention of ttie first person or any ttiird person in the 
environment. The second people^bserving device, having observed behavior of 
the first person, generates attention clues which it sends to the second attention 
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detection module. The second attention detection module analyses those 
attention clues and, where appropriate, generates one or more trigger signals and 
sends the trigger signal(s) to the second image capture device and/or first image 
capture device to trigger capturing of an image or Image sequence by the second 
and/or first image capture devices. 

[0087] Figure 13 is a schematic illustration of the interaction between 
first and second attention detection modules 1300, 1301. respectively worn by 
first and second host persons. 

t0088] First attention detection module 1300 performs attention 
detection processes 1. 2. 4, 10 based on a plurality of attention due signals. 1. 2. 
4, 10 respectively. The first attention detection module 1300 receives ftom ttie 
second attention detection module 130 data which are collected In an observer 
perspective mode by the second attention detection module. The second 
attention detection module 1301 monitors the first host person, and upon 
detec^ng an attention due of the first host person, signals to the first attention 
detection monitor 1300 infomiatoon concerning the first host person. The 
infonnation concerning the first host person is analyzed by the self perspedive 
analyzer of the first attention detection module 1300. 

[0089] Conversely, first attention detedion module 1300 monitors the 
second person. Module 1300, upon deteding an attention due of the second 
person, signals to tiie second attention detedion monitor 1301 attention dues 
concerning the second person. The second attention detection monitor 1301 
incorporates the information received fiiom ttie first attention detedion monitor, in 
a self perspedive analysis mode of the second attention detedion monitor. The 
second detedion monitor 1301 analyses attention dues of the second host 
person to detemiine whettier an Image signal should be generated for capturing 
an image using tiie second person's image capture device. 



